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Abstract. In this article we construct control policies that ensure bounded variance 
of a noisy marginally stable linear system in closed-loop. It is assumed that the noise 
sequence is a mutually independent sequence of random vectors, enters the dynamics 
affinely, and has bounded fourth moment. The magnitude of the control is required to 
be of the order of the first moment of the noise, and the policies we obtain are simple 
and computable. 



§1. Introduction 

Stabilization of stochastic linear systems with bounded control inputs has attracted 
considerable attention over the years. This is due to the fact that incorporating bounds 
on the control is of paramount importance in practical applications; suboptimal control 
strategies such as receding-horizon control [Chatterjee et al., 2009; Hokayem et al., 
2009], and rollout algorithms [Bertsekas, 2000], among others, were designed to in- 
corporate such constraints with relative ease, and have become widespread in appli- 
cations. However, the following question remains open: when is a linear system with 
possibly unbounded additive stochastic noise globally stabilizable with bounded inputs? 
In this article we shall provide sufficient conditions that give a positive answer to this 
question with minimal hypotheses. 

Bounded input control has a rich and important history in the control literature [Lin 
et al, 1996; Stoorvogel et al., 2007; Sussmann et al., 1994; Yang et al., 1997, 1992]. 
The deterministic version of the bounded input stabilization problem was solved com- 
pletely in a series of articles [Sussmann et al., 1994; Yang et al., 1992] culminating 
in [Yang et al., 1997]. It was demonstrated in [Yang et al., 1997] that global asymp- 
totic stabilization of a discrete-time linear system 

(*) x t+1 =Ax t +Bu t 

with bounded feedback inputs is possible if and only if the transition matrix has spec- 
tral radius at most 1, and the pair (A,B) is stabilizable with arbitrary controls. More- 
over, extensions to the output feedback case have appeared in [Bao et al., 2000; Chi- 
tour and Lin, 2003] . 

In the presence of affine stochastic noise the linear system (*) becomes x t+1 = 
Ax t +Bu t + w t , where (w t ) teNo is a collection of independent (but not necessarily iden- 
tically distributed) random vectors in K d with possibly inter-dependent components at 
each time t. With an arbitrary noise it is clearly not possible to ensure mean-square 
boundedness; for instance, if the noise has a spherically symmetric Cauchy distribution 
on K d , then given any initial condition x e R d , the second moment of x 1 does not 
even exist. Similarly, if the second moment of the noise becomes unbounded with time, 
it is not possible to control the second moment of the process (x t ) teN|) . It is necessary 
to assume, at least, that the noise has bounded variance. 
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Going beyond this necessary condition, it is not difficult to establish mean-square 
boundedness of such a system with bounded controls under the assumption that A is 
Schur stable, i.e., all eigenvalues of A are contained in the interior of the unit disk (the 
proof of this fact relies on standard Foster-Lyapunov techniques [Meyn and Tweedie, 
1993]). However, to the best of our knowledge, there is no proof that the same can be 
ensured for a marginally stable linear system. Results in this direction were reported 
in [Stoorvogel et al., 2007], but to the best of our understanding conclusive proofs of 
the facts reported in the present article are still missing in the literature. 

In this article, we develop easily computable bounded control policies for the case 
when A is marginally stable and (A,B) is stabilizable. Our policy is not anyway sta- 
tionary and is in general chosen from the class of finite fc-history-dependent and/or 
non-stationary policies. With respect to the case when A is orthogonal, it turns out 
that if the system is reachable in one step (i.e., rankB = the dimension of the state 
space), we do get stationary feedback policies. In the more general case when the sys- 
tem (*) is reachable in fc steps (with arbitrary controls), we propose a feedback policy 
for a sub-sampled system derived from the original one, which, for the actual system, 
turns out to be a fc-history-dependent policy. In fact, in this case we realize our policy 
as successive concatenations of a fixed fc-length policy. In the most general situation 
we propose a fc-history-dependent policy, where fc is now the reachability index of the 
particular subsystem of (A,B) for which the dynamics matrix is orthogonal. In all the 
mentioned cases, the length of the policy is at most equal to the dimension of the state 
space; memory requirements for even the most general case are, therefore, modest. 

Note that in our setting we do not assume that the noise is white. For our purposes 
the requirements on the noise are rather general, namely, the fourth moment of the 
noise should be uniformly bounded, and the noise vectors should be independent of 
each other (identical distribution at each time is not assumed). In particular, we do 
not assume Gaussian structure of the noise. It turns out that to ensure stabilization we 
need the controller to be sufficiently strong, in the sense that the control input norm 
bound should be bigger than a uniform bound on the first moment of the noise. 

Section 2 contains a precise statement of our result in the most general hypotheses 
(A marginally stable and (A,B) stabilizable), and a brief sketch of the proof. In Sec- 
tion 3, after some preliminary material, we prove the attainability of bounded second 
moment for a random walk, then we generalize the result under weaker and weaker 
hypotheses, finally culminating in the proof of the main theorem of Section 2. Section 
4 presents a numerical example illustrating our results, and Section 5 concludes the 
article with a conjecture. 

§2. Main result 

§2.1. Statement of the theorem. Consider the discrete-time linear system 

(2.1) x t+1 =Ax t +Bu t + w t , x = x, teN , 

where the following hold: x e R d is given; the state x t at time t takes values in R d ; 
A e R dxd , all the eigenvalues of A lie in the closed unit circle, and those eigenvalues A 
such that |A| = 1 have equal algebraic and geometric multiplicities; B e R dxm , and the 
control u t at time t takes values in R m ; (w t ) teNo is an R d -valued random process with 
mean zero and E [w t wj] = Q t . 

Our objective is to synthesize a fc-history-dependent control policy 1 n = (n: t ) teNo , 
consisting of successive concatenations of fc-length sequence Tr :/c-i := [^o> ' ' " > 
of maps, 7T ; : R d — > R m for i = 0, . . . , fc - 1, such that n t : R dxk — > R m is measurable, 



^ee §3.1 for definitions of policies. 
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u t := n t{ x t> x t-i>- ■ ■ > x t-k+i)> tne sequence (u t ) teN is bounded, and the state of the 
closed-loop system 

(2.2) x t+1 =Ax t +Bn t (x t ,x t _ 1 ,...,x t _ k+1 )+w t , x Q = x, teN , 

has bounded second-order moment. (To simplify the notation, we fix x_ k+1 = • • • = 
x_j = x .) The following is our main result: 

(2.3) . Theorem. Consider the system (2.1). Suppose that the pair(A,B) is stabilizable, 
and that sup teNo E[||w t || 4 ] < oo. Then there exist an R > and a deterministic k- 
history-dependent policy (7T t ) teNo , with k<d and ||^ t (0|| ^ R f° r every t, such that 
(PI) for every fixed x e K d the process (x t ) tsNo that solves the recursion (2.2) satisfies 

su PteN E *[||*t|| 2 ] <°°; and 
(P2) in the absence of the random noise the origin is asymptotically stable for the 
closed-loop system. 

§2.2. Sketch of the proof. Our proof is built in a series of steps, moving from simpler 
to progressively more complex systems. The starting point is the d-dimensional ran- 
dom walk x t+1 = x t + u t + w t . In this case we employ the main result of [Pemantle and 
Rosenthal, 1999] to design a policy that guarantees mean-square boundedness of the 
closed-loop system. We then consider the system x t+1 = Ax t +Bu t + w t , where u t is a 
d-dimensional control input, ranks = d, and A is orthogonal. With the help of a time- 
varying injective linear transformation this case is reduced to the d-dimensional ran- 
dom walk. The third case that we consider is that of the system x t+1 =Ax t +Bu t +w t , 
where u t e K m and A is orthogonal. This is reduced to the second case above with 
the aid of an injective linear transformation derived from the reachability matrix of the 
pair (A,B) (recall that by assumption the reachability matrix has rank d). Finally, the 
general case when A is just stable and (A,B) stabilizable is reduced to the third case 
with the observation that, in view of the stability hypothesis, A acts as an orthogonal 
map on its invariant subspace that corresponds to the eigenvalues that lie on the unit 
circle. 

Arguments for establishing mean-square boundedness of stochastic dynamical sys- 
tems typically rely on L 1 -bounded-ness of a Lyapunov-like functional of the system. 
The latter can be established in at least three different ways: The first is via the 
classical Foster-Lyapunov drift-conditions [Foss and Konstantopoulos, 2004; Meyn and 
Tweedie, 1993] and its various refinements; the second is via excursion-theoretic anal- 
ysis [Chatterjee and Pal, 2008] that relies primarily on the existence of certain su- 
permartingales as long as the process is outside some bounded set; the third is via 
martingale inequalities [Pemantle and Rosenthal, 1999], which applies to more gen- 
eral scalar-valued processes than Markov processes, and in the presence of bounded 
controls, provides the basic machinery for establishing our Theorem (2.3). 

§3. Proof of the main result 

§ 3.1. Preliminaries. Let N be the set of nonnegative integers {0,1,2,...}. The 
standard 2-norm on Euclidean spaces is denoted by ||-|| and the absolute value on 
K by |-|. In a Euclidean space we denote by B r the closed Euclidean ball of ra- 
dius r centered at the origin. If (y t ) teNo is a random process on a probability space 
(n,^, P), taking values in some Euclidean space, we let E x [c^(y s ;s = 0, 1, . . ., t)] de- 
note the conditional expectation of a measurable mapping <p of the process up to time 
t, given the initial condition y Q = x; in particular we define the rt-th moment of y t 
as E x [||y t || ]. We denote conditional expectation given a sub-cr-algebra 5' of $ as 
E[- \&\. For r > let sat r : R d — > B r be defined by sat r (y) := y if y e B r and 
sat r (y) := ry / y otherwise. Note that sat r (-) is not the component-wise saturation 
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function. Given matrices A e R dxd and B e K dxm we define the fc-step reachability 
matrix K k := [ B AB ■■■ A* _1 B ] . 

We specialize the general definition of a policy [Hernandez-Lerma and Lasserre, 
1996, Chapter 2] to our setting. A policy n := (7i t ) teN is a sequence of measur- 
able maps 7t t : R dxk — > R m for some fc e N, such that the control at time t is 
K t { x t> x t-iJ ■ ■ -> x t-k+i)- The policy n = (n t ) t( =^ a we have defined is also known as a 
deterministic k-history-dependent policy in the literature. A special case of these policies 
is a deterministic feedback policy or simply a feedback if k = 1 in the definition of a de- 
terministic history-dependent policy. Under deterministic feedback policies the closed- 
loop system is Markovian [Hernandez-Lerma and Lasserre, 1996, Proposition 2.3.5]. 
A further special case is when n t = f , a fixed measurable mapping / : K d — > K m for 
t e N ; this is known as a stationary feedback policy . 

(3.1). Lemma. LetB 1 , ■ ■ ■ ,B k be d x m matrices, M := [ B 1 ■■■ B k J , and o~ d de- 
note the minimum singular value of M. If rankM = d, then for all r > every 
vector v e R d belonging to B r can be expressed asv = X^i^i"/' wj£ ^ u i e ^ m an d 
\\ u i\\ — r<J d 1 - I* 1 P ar ti cu l ar , ifB e R dxd and ranks = d, then every vector v e R d 
belonging to B r can be expressed asv = Bu, where u e R d , \\u\\ < ro~~^ . 

Proof. rankM = d implies that km > d. Hence, M = [ B t ••• B k ] e R^xicm is 
a "flat" matrix. Let M = USV = U [ S J V T be a singular value decomposition 
of M, where £ = diag((7 1 ,...,cr d ). Since M has full rank, the matrix S is invertible. 
Hence every vector v e R d can be expressed as v = Mu, where u = M + v and M + = 
S" 1 1 

^ !7 T e R^mxd i s the Moore-Penrose pseudoinverse of M. Since U, V are or- 



V 

thogonal, for any p > we have inf|| u || =p ||Mu|| = inf|| V T u || =p 
inf||„n =p = pcr d . Hence, the image of B p under M contains B p(Jd , and if we 

choose p = ra^ 1 , then the image of B p under M contains B r . Notice that a^ 1 



U[ S ] V T u 



is also the greatest singular 




V 




U J v 


SU P||L7T v |[=r 






sup|| v || =r ||s 1 v|| = rcj d 1 . Summing up, every v e B r 

can be expressed as v = Mu, where u e R km and ||u|| < ra^ 1 . It remains to no- 
tice that u can be partitioned according to the partition of M, that is v = Mu = 
[ B 1 B 2 ■■■ B k ][u\ ■■■ ulY = £Li B i"i and the bound ||u|| < rcr" 1 im- 
plies ||u ; || < ra^ 1 for all i = 1 ••• k. □ 

§3.2. The d-dimensional random walk. At the core of our proof is the d-dimensional 
random walk: 

(3.2) x t+1 = x t + u t + w t , x = x, teN , 

with the state x t e R d , the control u t e K d with ||u t || ^ r for some r > 0, the noise 
process (w t ) teNo satisfies the following assumption: 

(3.3) . Assumption. 

o (w t ) tsNo are mutually independent d-dimensional random vectors (not necessarily 

identically distributed), 
o E[w t ] = 0, E [w t w]] = Q t for all t e N , 

o there exist C 4 > such that E [||w t || 4 ] ^ C 4 for all teN . 

Let C 1 := sup teNo E[ w t ]; this is well-defined because by Jensen's inequality we 
have C 1 ^ {/Q- Let (^ t ) teNo be the natural filtration of the system (3.2). Our proof 
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of Theorem (2.3) relies on the following (immediate) adaptation of the fundamental 
result [Pemantle and Rosenthal, 1999, Theorem 1]. 

(3.4) . Proposition. Let (£ t ) teN be a sequence of nonnegative random variables on 
some probability space (f2, P), and let (5 t )teN be any filtration to which (£ t ) teN is 
adapted. Suppose that there exist constants b > 0, and J,M < oo, such that £ ^ J, 
and for all t: 

(3.5) E[£ t+1 - ? t |? t ] ^ -b on the event {£ t > J}, and 

(3.6) E[|? t+1 -? t | 4 |£ ,...,£ t ] SSM. 

Then there exists a constant c = c(b, J, M) > such that sup E [^] ^ c. 

teN„ 

(3.7) . Lemma. Consider the system (3.2), and define £ t := x t L t e N . There exists 
a constant b > 0, such that for any r > C\ condition (3.5) hcJcis in closed-loop with 
the control u t = — sat r (x t ). 

Proq/: Fix t e N and r > C 1 . We have 

E[? t +i - £t|fft] = E [|| x t+i|| - INI |fft] = E[||* t + u t + w t || - ||x t || |ff t ] 
= E[||x t -sat r (x t ) + w t || - ||x t || \$ t ] 
^E[||x t -sat r (x t )|| + ||w t ||-||x t |||^]. 

Let J = r and b := r — C 1 . On the set {||x t || > ^} we have ||x t — sat r (x t )|| — ||x t || = — r. 
From the above we get, on the set {||x t || > J}> 

E[? t+ i - £ t |ff t ] ^ E[||x t - sat r (x t )|| + ||w t || - ||x t || |&] 
= -r + E[||w t ||] 

where b is positive by our hypothesis. The assertion follows. □ 

(3.8) . Lemma. Consider the system (3.2) and define £ t := ||x t ||; t e N . Then for the 
closed-loop system with u t = — sat r (x t ) there exists a constant M = M(C 4 ) > such 
that (3.6) holds. 

Proof. Fix r > C 1 . Applying the triangle inequality successively, we have 

|?t+i - ?tf = ||| x t+i|| _ || x t||| 4 ^ ||*t+i - ^t|| 4 = ||"t + w t \\ 4 ^ (r + ||w t ||) 4 , 
which leads to 

E[|? t+ i-? t rKo,---,? t ]^E[(r+||w t ||) 4 |? ,...,? t ]=E[(r+||w t ||) 4 ]. 

Since the fourth moment of w t is uniformly bounded, expanding the right-hand side 
above and applying Jensen's inequality shows that there exists some M = M(C 4 ) > 
such that E [(r + ||w t ||) 4 ] ^ M. The assertion follows. □ 

(3.9) . Proposition. For r > consider the system (3.2) under the deterministic sta- 
tionary feedback policy u t = — sat r (x t ): 

(3.10) x t+1 = x t - sat r (x t ) + w t , x = x, teN . 

Then for every r > C 1 the system (3.10) satisfies sup teNo E x [||x t || 2 ] ^ c for some 
c = c(x, Q) < oo. 

Proof. Let r = C 1 + b for some b > and J := max{r, ||x||}. Lemma (3.7) guarantees 
that (3.5) holds, and Lemma (3.8) shows that there exists an M > such that (3.6) 
holds. The assertion now is an immediate consequence of Proposition (3.4). □ 
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§3.3. The case of A orthogonal. Next we establish part (PI) of the main theorem in 
the particular case of A being orthogonal. 

(3.11) . Lemma. Consider the system y t+1 =Ay t + u t + w t , wherey t andu t takevalues 
in R d , A is orthogonal, and (w t ) teN satisfies Assumption (3.3). There exist a constant 
r > and a deterministic stationary policy n = (f,f,- m sucn that ||/(y)|| ^ r for all 
y e R d and t e N , and the closed-loop system 

(3.12) y t+ i=Ay t +fiy t ) + w t 
under this policy satisfies sup tsNo [||y t || 2 ] < oo. 

Proof. Consider the process (z t ) teN denned by z t := (A 7 ) 1 y t . The second moment of 
z t is the same as that of y t due to orthogonality of A: 

E*[|N| 2 ] =E X [||(A T ) £ y t || 2 ] =E x [y £ MWy t ] =E x [y t T y t ] =E x [||y t || 2 ]. 
Now we have 

(3.13) z t+1 = m +1 yt+i = (A r Y y t + G4 T ) t+1 u t + (A T ) t+1 w t = z t + u t + w t , 

where the mapping u t < — > u t := (A T ) t+1 u t is isometric and invertible, and (w t ) t€N 
defined by w t := G4 T ) t+1 w t , is a sequence of zero-mean, independent (although 
in general not identically distributed) random vectors, with fourth moment given 
by E[||w t || 4 ] = E[||G4 T ) t+1 w t || 4 ] = E[||w t || 4 ] ^ C 4 . Due to Proposition (3.9), 
there exists a constant r such that the closed-loop system (3.13) under the policy 
u t = — sat r (z t ) =: /(z t ) has bounded second moment. Consequently, the original sys- 
tem (3.12) has bounded second moment under the policy 

u t =A t+1 u t =A t+1 f(z t ) = -A t+1 sat r (G4 T ) f y t ) =: / t (y t ). 

Noting that for any orthogonal matrix A we have sat r (Ay) = Asat r (y), we arrive at 
u t = / t (y t ) = — Asat r (y t ) =: /(y £ ), which is indeed a stationary feedback. Moreover, 
since ||Asat,.(y t )|| < r, we have ||/(y t )|| < r. □ 

In the following we will consider a nonstationary policy obtained by successive con- 
catenations of a fc-length policy (/ >/i> - "A-i) acting on the "sub-sampled" process 
(x nfc ) neN . More precisely, our policy has the form iz t = Bf t mod k( x (t^k)k) where the 
"-r" symbol denotes integer division and "mod" its remainder. In words, we break 
the time line into segments of length k, and within each segment we let the controls 
be given by / ,/i, • • -fk-i> applied in this order always to the first state observed in 
the segment. For example, x 1 = x + B/ (x ) + w , x 2 = x 1 + B/^Xq) + w 1 , 
x k = x fc _j +Bf k _ 1 (x )+w k _ 1 , x k+1 = x k +Bf [x k ) + w k , x k+2 = x k+1 +Bf 1 ix k ) + w k+1 , 
and so on. 

(3.14) . Lemma. Consider the system 

(3.15) x t+1 =Ax t +Bu t + w t , 

where x t takes values in R d , u t takes values in R m , A is orthogonal, the pair (A,B) is 
reachable in k steps (i.e., rank7vL fc = d, where lZ k = [ B AB ••• A k ~ l B J J, and 
(Wt)teN satisfies Assumption (3.3). Then there exist a constant p > and a policy 

n = (/o,/i, ■ • ■ A-i,/oj/ij ■ ■ • ) su ch that \\fi( x )\\ ^ P f° r a ^ x e an d the closed- 
loop system 

(3.16) x t+1 =Ax t +Bf t 
under this policy satisfies sup teN() E x [||x t || 2 ] < oo. 
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Proof. Let t e N and consider the evolution of (3.15) from time ik to time (t + l)fc: 



(3.17) x [r+1)k =A k x zk + K k 



(T+l)fc-l 



U 



Tk 



k-1 



+ 1 l w Tk+i = Ax zk + u z + w T , 



i=0 



where w T := ^ i= g A k 1 V Tfc+j is a random vector with mean zero and bounded fourth 
moment. Since lZ k has full rank, Lemma (3.1) implies that for arbitrary r > 0, any u T 
in B r can be expressed as u T = A fc_1_! Bu Tfc+; , where ||u tS . +1 -|| < rcr^ 1 and a d is 
the smallest singular value of lZ k . But from Lemma (3.11) we know that there exists 
a particular r > such that, under the stationary policy u z = f(x zk ) = —Asat r (x zk ), 
the "sub-sampled" system (3.17) has bounded second moment, and ||u T || < r. There- 
fore, if we choose p = ra^ 1 , there exists a constant c = c(x, C 1; C 4 ) > such that 
su Pt€N E x [||x T k|| 2 ] ^ c. It follows from the system dynamics that for n = 0, . . . , fc — 1 , 

E x [||x Tfc+n || 2 ] ^ 2(c + n 2 r 2 o- 1 (B) 2 )+fc max trQ zk+n 

11 11 n=0,...,k— 1 

sj 2(c + nVCTjCB^+fc^/Q. 

where the last step follows from Jensen's inequality. Since the right-hand side above 
constitutes a uniform bound, this proves the assertion. □ 



(3.18). Remark. The actual policy for (3.15) is 



(T+l)fc-l 



-K k AsatXx zk ). The 



proof above shows that all the inputs u^ +1 ~ )k _ 1 , ■ ■ ■ ,u rk can be computed at time ik 
in order to counteract the future effect of the current state, i.e. Ax zk , and ignoring the 
effect of the noise for the following fc steps. In the particular case when B e R dxd has 
full rank, m = d, and obviously fc = 1, the above policy is stationary, and in particular 
it has the form: u t =/(x t ) = — B _1 Asat r (x t ). Once again we have ||iz t || < ra^ 1 , where 
this time a d is the smallest singular value of B. < 

§3.4. Proof of Theorem (2.3). 

Proof. Consider the system (2.1), with (A,B) stabilizable and (w t ) teN with bounded 
fourth moment. If A is Schur stable (that is, all the eigenvalues of A belong to the 
interior of the unit disk), the system with zero input has bounded second moment and 
is asymptotically stable, and there is nothing to prove. Otherwise, there exists a change 
of base in the state-space that brings the original pair (A, B) to a new pair (A, B) , where 
A is in real Jordan form [Horn and Johnson, 1990, p. 150]. In particular, choosing 
a suitable ordering of the Jordan blocks, we can ensure that the pair (A,B) has the 
form ( [ q 1 A22 ] , [ B l 2 ] ) , where A n is Schur stable, and A 22 has its eigenvalues on the 
unit circle. Due to the stability hypothesis (the algebraic and geometric multiplicities 
of the eigenvalues of A 22 are equal), A 22 is therefore block-diagonal with elements on 
the diagonal being either ±1 or 2 x 2 rotation matrices. As a consequence, A 22 is 
orthogonal. Moreover, since (A,B) is stabilizable, the pair (A 22 ,B 2 ) must be reachable 
in a number of steps fc < d which depends on the dimension of A 22 and the structure of 
(A 22 ,B 2 ), since it contains precisely the modes of A which are not asymptotically stable. 
Summing up, we can reduce the original system x t+1 = Ax t + Bu t + w t to the form 

[ X J*>~\ = [a^ 2 '] + + [I/ 2 ']' where A n is Schur stable, A 22 is orthogonal, 

(A 22 ,B 2 ) is reachable, and ^ |^ w ( 2 , j J is derived from (w t ) teNo by means of linear 

transformations. We know that since A n is Schur stable, the noise (wj 1 ')^ has 



8 



F. RAMPONI, D. CHATTERJEE, A. MILI AS - ARGEITI S , R HOKAYEM, AND J. LYGEROS 



bounded second moment, and the control inputs (u t ) teN are bounded, then the x^ 
sub-system is mean-square bounded under any Markovian control [Chatterjee et al., 
2009, §4] . Therefore, if under some bounded policy the x® sub-system is mean-square 
bounded, the original system will also be mean-square bounded under the same policy 
Thus, at least for the proof of (PI), it suffices to restrict our attention to the subsystem 
described by the pair {A.22^2) • Suppose that this subsystem is reachable in a certain 
number fc < d of steps. 

The proof of (PI) coincides with the proof of Lemma (3.14), where we obtain 



rcr d for r > C 1 and a d is the smallest singular value of TZ k . (Here, lZ k = 

j 2 ^22^2 ■ • ■ A 22 ] ■) As the control authority required in the claim of the 
theorem, we choose precisely R = p. 

To prove (P2), notice that for the closed-loop "sub-sampled" system without noise 



[ B 2 



under the policy u t 
(3.19) 



U+Asat r (x[ 2) ), where A = A k 22 , it holds: 



x 



As long as jr, J is outside B r , 



(2) 

(t+1)/c 
(2) 



Ax%-Asat r {x%). 



< r. 



steps it must hold 
definition of sat r (-) we have x^ 



(t+1)/c 



r. Hence, in a finite number of 



When for some t we have 

.(2) _ 



for all t > t . Hence, 



> =f — 0, and consequently jc ^ 
the state of the closed-loop "sub-sampled" system converges to zero in finite time for 
any initial condition. Then, according to the chosen policy, for all t > f we have 



U (T+l)fc-l 



and u T 



U (t+1)/c-1 



0, and consequently, for 



t > t and t?c < t < (t + l)fc we also have x[ 2) = 0, that is, x\ A> = V t > ffc, which 
proves (P2) for the subsystem (A 2 2,B 2 ) of our system (2.1). 

Finally, to extend the result (P2) to the general case (where A ■ 
suffices to note that, since for t > xfc it also holds u t = 0, from the time f fc onwards 



diag(A n ,A 22 )), it 



the subsystem (A n ,B 1 ) is in open loop. Since we imposed A n to be Schur stable, the 
state x[ of the latter converges to zero as t — » 00. This proves the theorem. □ 



§4. Numerical Example 

An example follows, which shows that our nonlinear policy is readily computable, 
and effective in bounding the state of a stable linear system in the mean square. 
We executed 1000 runs of simulation of the system x t+1 = Ax t + Bu t + w t , where 



A - 



cos {pi — sin ^3 

sin cos <p r 









0.5 

0.9 





- 1 " 


,B = 


000 



, with ip t = 0.8, x = [ 10 20 30 40 ] T , 



and 



where w t is a Gaussian white noise with variance I 4 . This system is marginally stable 
and, as is easily seen, the 2-dimensional subsystem with eigenvalues on the unit circle 
is reachable in 2 steps, whereas the 2-dimensional Schur-stable subsystem is not reach- 
able at all. The control authority R was chosen approximately equal to 3.6 according 
to a rough estimate of C 1 = E x [||w t ||] . It should be noticed that smaller values of R 
are also sufficient to stabilize the system. Figure 4.1 shows the empirical average of 
||x t || 2 over the 1000 runs, respectively with disabled control, with the chosen control 
authority, and with one tenth of the chosen control authority. 



§5. A Conjecture 

We conjecture that if the noise has bounded variance, then given any arbitrary posi- 
tive uniform upper-bound on the norm of the control, there exists a stationary feedback 
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Figure 4.1. Empirical average of ||x t || 2 over 1000 runs. 

policy such that the closed-loop system is mean-square bounded. It appears to us that 
a proof of this conjecture will require substantially new and nontrivial techniques. 
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